User Interfaces for Mobile Augmented Reality Systems

User Interfaces for Mobile Augmented Reality Systems Tobias Hans Höllerer In this dissertation, we present typical components of, useful services associated with, and user interactions possible with mobile augmented reality systems, based on a comprehensive series of hardware and software infrastructures and application prototypes we developed. We define a practical taxonomy of user interface components for such systems and establish methodology for adaptive mobile augmented reality interfaces that dynamically rearrange themselves in response to changes in user context. The research contributions to the state-of-the-art in augmented reality begin with the author’s participation in the design of the ”Columbia Touring Machine” in 1997, the first example of an outdoor mobile augmented reality system, and his lead in developing later prototypes. We develop a series of hardware and software infrastructures for prototyping mobile augmented reality applications that allow multiple users to participate in collaborative tasks taking place indoors and outdoors. We present exploratory user interfaces for many different applications and user scenarios, including the Situated Documentaries application framework for experiencing spatially distributed hypermedia presentations. Based on these explorations, we develop a taxonomic categorization of mobile augmented reality interface components and their properties. Virtual and real world objects alike are considered part of the interface. We tag each component with information about its purpose, its intrinsic properties, its relationship to other objects, and its capabilities and flexibility with regard to various manipulations. Mobile augmented reality has until now faced a significant challenge: the complexity of the augmented views rapidly increases when many virtual objects fight for screen space to annotate physical entities in the dynamic views of multiple fast-paced roaming users. Responding to this, we develop user interface management techniques for mobile augmented reality. A rule-based reasoning architecture uses the taxonomic data classification mentioned above to automatically rearrange augmented reality views in dynamic situations; for example to remove clutter in the augmented view of the world or to react to infrastructural context changes, such as variations in tracking accuracy.